Information processing apparatus, information processing method and program

ABSTRACT

There is provided an information processing apparatus and information processing method to implement a more natural and flexible behavior plan of an autonomous mobile object, the information processing apparatus including a behavior planner configured to plan a behavior of an autonomous mobile object based on estimation of circumstances, wherein the behavior planner is configured to, based on the circumstances that are estimated and multiple sets of needs that are opposed to each other, determine the behavior to be executed by the autonomous mobile object. The information processing method includes, by a processor, planning a behavior of an autonomous mobile object based on estimation of circumstances, wherein the planning includes, based on the circumstances that are estimated and multiple sets of needs that are opposed to each other, determining the behavior to be executed by the autonomous mobile object.

FIELD

The present disclosure relates to an information processing apparatus,an information processing method and a program.

BACKGROUND

In recent years, various apparatuses with learning functions have beendeveloped. The apparatuses include autonomous mobile objects, such as arobot that performs autonomous operations based on estimatedcircumstances. For example, Patent Literature 1 discloses a footedmobile robot that performs autonomous operations and expresses emotionscorresponding to the circumstances.

CITATION LIST Patent Literature

Patent Literature 1: Japanese Laid-open Patent Publication No.2003-71763

SUMMARY Technical Problem

Operations of an autonomous mobile object like that described in PatentLiterature 1 are executed according to a behavior plan based onestimation of circumstances. When uniform behaviors are determined forestimated circumstances, simplistic operations patterns are caused andthus a situation in which the interest of the user in the autonomousmobile object lowers is also assumed.

The present disclosure thus proposes an information processingapparatus, an information processing method and a program that are noveland improved and that make it possible to implement a more natural andflexible behavior plan of an autonomous mobile object.

Solution to Problem

According to the present disclosure, an information processing apparatusis provided that includes: a behavior planner configured to plan abehavior of an autonomous mobile object based on estimation ofcircumstances, wherein the behavior planner is configured to, based onthe circumstances that are estimated and multiple sets of needs that areopposed to each other, determine the behavior to be executed by theautonomous mobile object.

Moreover, according to the present disclosure, an information processingmethod is provided that includes: by a processor, planning a behavior ofan autonomous mobile object based on estimation of circumstances,wherein the planning includes, based on the circumstances that areestimated and multiple sets of needs that are opposed to each other,determining the behavior to be executed by the autonomous mobile object.

Moreover, according to the present disclosure, a program is providedthat causes a computer to function as an information processingapparatus comprising a behavior planner configured to plan a behavior ofan autonomous mobile object based on estimation of circumstances,wherein the behavior planner is configured to, based on thecircumstances that are estimated and multiple sets of needs that areopposed to each other, determine the behavior to be executed by theautonomous mobile object.

Advantageous Effects of Invention

According to the disclosure described above, it is possible to implementa more natural and flexible behavior plan of an autonomous mobileobject.

The aforementioned effect is not necessarily restrictive and, togetherwith the aforementioned effect, or instead of the aforementioned effect,another effect that can be known from any one of the effects representedin the specification or the specification may be derived.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an exemplary hardware configuration ofan autonomous mobile object according to an embodiment of thedisclosure.

FIG. 2 is an exemplary configuration of an actuator that the autonomousmobile object includes according to the embodiment of the disclosure.

FIG. 3 is a diagram for describing an operation of the actuator of theautonomous mobile object according to the embodiment of the disclosure.

FIG. 4 is a diagram for describing the operation of the actuator of theautonomous mobile object according to the embodiment of the disclosure.

FIG. 5 is a diagram for describing a function of a display that theautonomous mobile object includes according to the embodiment of thedisclosure.

FIG. 6 is a diagram illustrating an exemplary operation of theautonomous mobile object according to the embodiment of the disclosure.

FIG. 7 is a diagram illustrating an exemplary system configurationaccording to the embodiment of the disclosure.

FIG. 8 is a diagram illustrating an exemplary functional configurationof the autonomous mobile object according to the embodiment of thedisclosure.

FIG. 9 a diagram illustrating an exemplary functional configuration ofan information processing server according to the embodiment of thedisclosure.

FIG. 10 is a diagram for describing teaching an action using an userinterface according to a first embodiment of the disclosure.

FIG. 11 is a diagram for describing teaching according to physicalbending and stretching operations on a joint part according to the firstembodiment.

FIG. 12 is a diagram for describing teaching according to movements ofan animal object of which images are captured according to the firstembodiment.

FIG. 13 is a diagram for describing trimming based on a range of motionaccording to the embodiment.

FIG. 14 is a diagram for describing teaching that specifies relativepositions relating to multiple autonomous mobile objects according tothe first embodiment.

FIG. 15 is a diagram for describing editing control sequence dataaccording to the first embodiment.

FIG. 16 is a diagram for describing cause circumstances according to thefirst embodiment.

FIG. 17 is a diagram for describing cause circumstances according to thefirst embodiment.

FIG. 18 is a diagram for describing transmission of control sequencedata between the autonomous mobile objects according to the firstembodiment.

FIG. 19 is a flowchart representing a flow of control of the autonomousmobile object 10 relating to teaching by physical bending and stretchingoperations of a joint part that are performed by a user according to thefirst embodiment.

FIG. 20 is a flowchart representing a flow of control of an autonomousmobile object using movements of an animal object of which images arecaptured as teaching according to the first embodiment.

FIG. 21 is a diagram for describing a behavior plan based on whetherthere is detection of a user according to a second embodiment of thedisclosure.

FIG. 22 is a diagram for describing a behavior plan prioritizingreduction of power consumption according to the second embodiment.

FIG. 23 is a diagram illustrating an exemplary behavior plan satisfyingboth self-preservation needs and self-esteem needs according to thesecond embodiment.

FIG. 24 is a diagram for describing a behavior plan based on a distancebetween a user and an autonomous mobile object according to the secondembodiment.

FIG. 25 is a diagram illustrating an exemplary behavior plan based on achange in environmental condition according to the second embodiment.

FIG. 26 is a diagram illustrating an exemplary behavior plan based on achange in environmental condition according to the second embodiment.

FIG. 27 is a diagram illustrating an exemplary behavior plan based on achange in environmental condition according to the second embodiment.

FIG. 28 is a diagram for describing an exemplary behavior plan based ona control mode of the autonomous mobile object according to the secondembodiment.

FIG. 29 is a flowchart representing a flow of a behavior plan accordingto the embodiment.

FIG. 30 is a diagram for describing representation of a recommendedbehavior according to a third embodiment of the disclosure.

FIG. 31 is a conceptual view to describe a behavior plan based on arecommended behavior according to the third embodiment.

FIG. 32 is a diagram for describing representation of a recommendedbehavior based on a degree of love of a user according to the thirdembodiment.

FIG. 33 is a diagram for describing collection of control sequence dataaccording to the third embodiment.

FIG. 34 is a diagram illustrating an exemplary user interface of aclient application according to the third embodiment.

FIG. 35 is a diagram for describing additional registration in an objectrecognition dictionary according to the third embodiment.

FIG. 36 is a diagram for describing additional registration in an audiorecognition dictionary according to the third embodiment.

FIG. 37 is a diagram for describing a function of recommendingmaintenance according to the third embodiment.

FIG. 38 is a flowchart representing a flow of representation ofrecommended behavior by an information processing server according tothe third embodiment.

FIG. 39 is a diagram illustrating an exemplary hardware configuration ofas information processing server according to an embodiment of thedisclosure.

DESCRIPTION OF EMBODIMENTS

With reference to the accompanying drawings, preferable embodiments ofthe disclosure will be described. In the description and drawings,components that substantially have the same function are dented with thesame numbers and redundant description is thus omitted.

Description will be given in the following order.

-   1. Configuration    -   1.1. Overview of autonomous mobile object 10    -   1.2. Exemplary hardware configuration of autonomous mobile        object 10    -   1.3. Exemplary system configuration    -   1.4. Exemplary functional configuration of autonomous mobile        object 10    -   1.5. Exemplary functional configuration of information        processing server 20-   2. First Embodiment    -   2.1. Overview    -   2.2. Generation of control sequence data    -   2.3. Flow of control-   3. Second Embodiment    -   3.1. Overview    -   3.2. Specific example of behavior plan    -   3.3 Flow of control-   4. Third Embodiment    -   4.1. Overview    -   4.2. Representation of recommended behavior    -   4.3. Additional registration in recognition dictionary    -   4.4. Recommendation of maintenance    -   4.5. Control flow-   5. Exemplary hardware configuration-   6. Summary

1. Configuration 1.1. Overview of Autonomous Mobile Object 10

First of all, an overview of an autonomous mobile object 10 according toan embodiment of the disclosure will be described. The autonomous mobileobject 10 according to the embodiment of the disclosure is aninformation processing apparatus that executes circumstances estimationbased on collected sensor information and autonomously selects andexecutes various operations corresponding to the circumstances. Unlike arobot that simply performs an operation according to an instructioncommand from a user, the autonomous mobile object 10 has acharacteristic in autonomously executing an operation that is estimatedas an optimum one according to each set of circumstances.

For this reason, the autonomous mobile object 10 according to theembodiment of the disclosure sometimes purposely does not execute anoperation corresponding to an instruction from the user or executesanother behavior different from the operation. For example, the casewhere an operation corresponding to an instruction from the user isperformed, the case where the safety of the user and the autonomousmobile object 10 or of the surrounding environment is impaired, and thecase where the autonomous mobile object 10, for example, prioritizesother needs (instinct), such as power charging, apply to theabove-described circumstances.

The autonomous mobile object 10 sometimes purposely does not follow aninstruction from the user to try to cause an interest of the user or tryto transmit the feeling of the autonomous mobile object 10 and thecondition of hardware to the user.

On the other hand, the autonomous mobile object 10 has strong needs(instinct) for being loved by the user. The autonomous mobile object 10repeatedly executes an operation corresponding to an instruction fromthe user in order to please the user and learns an operation that theuser likes and autonomously executes the operation even withoutinstruction.

As described above, as animals including humans do, the autonomousmobile object 10 according to the embodiment of the disclosurecomprehensively determines needs and feelings and the surroundingenvironment and determines and executes autonomous operations. As forthe aspect described above, the autonomous mobile object 10 isdefinitely different from a passive apparatus that executes acorresponding operation or process based on an instruction.

The autonomous mobile object 10 according to the embodiment of thedisclosure may be an autonomous mobile robot that autonomously moves ina space and executes various operations. The autonomous mobile object 10may be, for example, an autonomous mobile robot with a shape imitatingan animal, such as a human or a dog, and operational ability. Theautonomous mobile object 10 may be, for example, a vehicle with anability to communicate with users or may be another apparatus. Theshape, ability and level of needs, etc., of the autonomous mobile object10 of the disclosure can be designed as appropriate according to thepurpose and role.

1.2. Exemplary Hardware Configuration of Autonomous Mobile Object 10

An exemplary hardware configuration of the autonomous mobile object 10according to the embodiment of the disclosure will be described. Thecase where the autonomous mobile object 10 is a dog-type quadruped robotwill be described as an example below.

FIG. 1 is a diagram illustrating an exemplary hardware configuration ofthe autonomous mobile object 10 according to the embodiment of thedisclosure. As illustrated in FIG. 1, the autonomous mobile object 10 isa dog-type quadruped robot including a head part, a torso part, four legparts, and a tail part. The autonomous mobile object 10 includes twodisplays 510 in the head part.

The autonomous mobile object 10 includes various sensors. The autonomousmobile object 10, for example, includes microphones 515, cameras 520, aToF (Time of Flight) sensor 525, a human detection sensor 530, rangesensors 535, touch sensors 540, an illuminance sensor 545, sole buttons550, and inertia sensors 555.

Microphone 515

The microphone 515 has a function of collecting surrounding sounds. Thesounds include utterance of the user and surrounding environmentalsounds. The autonomous mobile object 10 may, for example, include fourmicrophones on the head part. Including the microphones 515 makes itpossible to sensitively collect sounds occurring around and realizelocalization of a sound source.

Camera 520

The camera 520 has a function of capturing an image of the user and thesurrounding environment. The autonomous mobile object 10 may include twowide-angle cameras at the tip of nose and the waist. In this case, thewide-angle camera that is arranged at the tip of nose captures imagescorresponding to the front field of view of the autonomous mobile object(that is, the field of view of dog) and the wide-angle camera at thewaist captures images of the surrounding area mainly of the upper area.The autonomous mobile object 10, for example, is able to extract afeature of the ceiling based on images captured by the wide-angle camerathat is arranged at the waist and realize SLAM (SimultaneousLocalization and Mapping).

ToF Sensor 525

The ToF sensor 525 has a function of detecting a distance to an objectthat is present in front of the head part. The ToF sensor 525 isincluded at the tip of nose of the head part. The ToF sensor 525 enablesaccurate detection of distances to various objects and thus enablesrealization of an operation corresponding to the relative positions withrespect to objects including the user and obstacles.

Human Detection Sensor 530

The human detection sensor 530 has a function of sensing presence of auser or a pet that the user rears. The human detection sensor 530 isarranged on, for example, the chest. The human detection sensor 530senses an animal object that is present ahead and thus enablesrealization of operations corresponding to feelings, such as interest,fear, and surprise, to the animal object.

Range Sensor 535

The range sensor 535 has a function of acquiring the circumstances ofthe floor surface in front of the autonomous mobile object 10. The rangesensor 535 is, for example, arranged on the chest. The range sensor 535enables accurate detection of a distance to an object that is present onthe floor surface in front of the autonomous mobile object 10 and thusenables realization of an operation corresponding to a relative positionwith respect to the object.

Touch Sensor 540

The touch sensor 540 has a function of sensing contact by the user. Thetouch sensor 540 is, for example, arranged in a part, such as the top ofhead, the bottom of chin, or the back where the user highly likelytouches the autonomous mobile object 10. The touch sensor 540 may be,for example, a capacitive or pressure-sensitive touch sensor. The touchsensor 540 enables sensing an act of contact by the user, such astouching, petting, hitting or pushing, and thus enables an operationcorresponding to the act of contact.

Illuminance Sensor 545

The illuminance sensor 545 detects an illuminance of a space in whichthe autonomous mobile object 10 positions. The illuminance sensor 515,for example, may be arranged at the base of the tail part behind thehead part. The illuminance sensor 545 enables detection of surroundingbrightness and thus enables execution of an operation corresponding tothe brightness.

Sole Button 550

The sole button 550 has a function of sensing whether the bottom of aleg part of the autonomous mobile object 10 contacts the floor. The solebuttons 550 are thus arranged respectively on parts corresponding to thepaw pads of the four leg parts. The sole button 550 makes it possible tosense contact or non-contact between the autonomous mobile object 10 andthe floor surface and thus, for example, know that the autonomous mobileobject 10 is lifted in the arms of the user.

Inertia Sensor 555

The inertia sensor 555 is a six-axis sensor that detects physicalquantities, such as a speed, an acceleration and a rotation, or the headpart and the torso part. In other words, the inertia sensor 555 detectsaccelerations and angular speeds of an X-axis, a Y-axis and a Z-axis.The inertia sensors 555 are arranged respectively at the head part andthe torso part. The inertia sensor 555 enables accurate detection ofmovements of the head and torso part of the autonomous mobile object 10and thus enables realization of operational control corresponding to thecircumstances.

Exemplary sensors of the autonomous mobile object 10 according to theembodiment of the disclosure have been described. The above-describedconfiguration given above using FIG. 1 is an example only and theconfigurations of sensors the autonomous mobile object 10 can includeare not limited thereto. The autonomous mobile object 10 may furtherinclude, in addition to the above-described configuration configuration,various communication devices including a temperature sensor, ageomagnetic sensor, and a GNSS (Global Navigation Satellite System)signal receiver. The configurations of sensors of the autonomous mobileobject 10 may be flexibly varied according to the specification andoperation.

An exemplary configuration of joint parts of the autonomous mobileobject 10 according to the embodiment of the disclosure will bedescribed. FIG. 2 is an exemplary configuration of actuators 570 thatthe autonomous mobile object 10 includes according to the embodiment ofthe disclosure. The autonomous mobile object 10 according to theembodiment of the disclosure has 22 degrees of freedom of rotation intotal including, in addition to degrees of freedom of rotation inrotating parts illustrated in FIG. 2, two degrees of freedom of rotationin each of the pair of ears and the tail part, and one degree of freedomof rotation in the mouth.

For example, the autonomous mobile object 10 has three degrees offreedom in the head part and thus enables both operations of nodding andtilting the head. The autonomous mobile object 10 enables realization ofnatural and flexible operations more close to those of a real dog byreproducing swing operations of the waist.

The autonomous mobile object 10 according to the embodiment of thedisclosure may realize the aforementioned 22 degrees of freedom ofrotation by combining a single-axis actuator and a two-axis actuator.For example, single-axis actuators may be used for the elbows and kneesof the leg parts and two-axis actuators may be used for the shouldersand the tails of thighs, respectively.

FIGS. 3 and 4 are diagrams for describing operations of the actuator 570of the autonomous mobile object 10 according to the embodiment of thedisclosure. Referring to FIG. 3, by rotating an output gear using amotor 575, the actuator 570 enables a movable arm 590 to be driven at agiven rotational position and a rotational speed.

Referring to FIG. 4, the actuator 570 according to the embodiment of thedisclosure includes a rear cover 571, a gear box cover 572, a controlboard 573, a gear box base 574, the motor 575, a first gear 576, asecond gear 577, an output gear 578, a detection magnet 579, and twobearings 580.

The actuator 570 according to the embodiment of the disclosure may be,for example, a magnetic svGMR (spin-valve Giant Magnetoresistive). Thecontrol board 573 causes the motor 575 to rotate according to thecontrol by a main processor, so that power is transmitted to the outputgear 578 via the first gear 576 and the second gear 577 and accordinglythe movable arm 590 can be driven.

The position sensor that the control board 573 includes detects an angleof rotation of the detection magnet 579 that rotates in synchronizationwith the output gear 578, thereby enabling accurate detection of anangle of rotation of the movable arm 590, that is, a rotationalposition.

The magnetic svGMR employs a non-contact system and thus has excellentdurability and the magnetic svGMR is used in a GMR saturated area andthus has an advantage in that effects of signal variations resultingfrom variations in distance between the detection magnet 579 and theposition sensor are small.

The exemplary configuration of the actuator 570 of the autonomous mobileobject 10 according to the embodiment of the disclosure has beendescribed. The above-described configuration enables accurate control onbending and stretching operations of the joint parts that the autonomousmobile object 10 has and thus enables accurate detection of therotational positions of the joint parts.

Referring to FIG. 5, the function of the display 510 that the autonomousmobile object 10 according to the embodiment of the disclosure includeswill be described. FIG. 5 is a diagram for describing the function ofthe display 510 of the autonomous mobile object 10 according to theembodiment of the disclosure.

Display 510

The display 510 has a function of visually expressing motions of eyesand feelings of the autonomous mobile object 10. As illustrated in FIG.5, the displays 510 are enables expression of motion of eyeballs,pupils, and eyelids corresponding to feelings and operations. Thedisplays 510 purposely do not display texts and symbols and images notrelating to eyeball movements, thereby producing natural operationsclose to those of an existing animal, such as a dog.

As illustrated in FIG. 5, the autonomous mobile object 10 includes twodisplays 510 r and 510 l corresponding respectively to the right eye andthe left eye. The displays 510 r and 510 l are, for example, implementedusing independent two OLEDs (Organic Light Emitting Diodes). The OLEDsenable reproduction of curved surfaces of the eyeballs and enable morenatural appearance compared to the case where a pair of eyeballs isexpressed using a single planar display or the case where the twoeyeballs are expressed respectively using two independent planardisplays.

As described above, the displays 510 r and 510 l enable accurate andflexible expression of gaze and feelings of the autonomous mobile object10. The user is able to instinctively know the condition of theautonomous mobile object 10 from operations of eyeballs that aredisplayed on the displays 510.

The exemplary hardware configuration of the autonomous mobile object 10according to the embodiment of the disclosure has been described. Theabove-described configuration enables realization of operations andexpression of feelings more close to those of an existing living thingby accurately and flexibly controlling operations of joint parts andeyeballs of the autonomous mobile object 10 as illustrated in FIG. 6.FIG. 6 is a diagram illustrating exemplary operations of the autonomousmobile object 10 according to the embodiment of the disclosure, and FIG.6 illustrates the external structure of the autonomous mobile object 10in a simplified manner in order to give description focusing onoperations of the joint parts and eyeballs of the autonomous mobileobject 10. Similarly, in the following description, the externalstructure of the autonomous mobile object 10 is sometimes illustrated ina simplified manner; however, the hardware configuration and theappearance of the autonomous mobile object 10 according to theembodiment of the disclosure is not limited the example illustrated inthe drawings and they may be designed appropriately.

1.3. Exemplary System Configuration

An exemplary system configuration according to an embodiment of thedisclosure will be described. FIG. 7 is a diagram illustrating anexemplary system configuration according to the embodiment of thedisclosure. Referring to FIG. 7, the information processing systemaccording to the embodiment of the disclosure includes a plurality ofautonomous mobile objects 10 and an information processing server 20.The autonomous mobile objects 10 and the information processing server20 are connected to each other and the autonomous mobile objects 10 areconnected to each other such that they are able to communicate with eachother via a network 30.

Autonomous Mobile Object 10

The autonomous mobile object 10 according to the embodiment of thedisclosure is an information processing apparatus that executescircumstances estimation based on the collected sensor information andautonomously selects and executes various operations corresponding tothe circumstances. As described above, the autonomous mobile object 10according to the embodiment of the disclosure may be, for example, anautonomous mobile robot that has a shape imitating an animal, such as ahuman or a dog, and operational ability.

Information Processing Server 20

The information processing server 20 according to the embodiment of thedisclosure is an information processing apparatus that is connected tothe autonomous mobile objects 10 and that has a function of collectingvarious types of information from the autonomous mobile objects 10. Theinformation processing server 20 is, for example, capable of performinganalysis relating to the condition of hardware of the autonomous mobileobject 10 and the degree of love of the user for the autonomous mobileobject 10 from the sensor information that is collected by theautonomous mobile objects 10.

The information processing server 20 has a function of, based on thecircumstances that are estimated by the autonomous mobile object 10,representing a recommended behavior that the autonomous mobile object 10should take in the circumstances. The information processing server 20may transmit control sequence data for casing the autonomous mobileobject 10 to implement the recommended behavior to the autonomous mobileobject 10. The above-described function of the information processingserver 20 will be described in detail separately.

Network 30

The network 30 has a function of connecting the autonomous mobileobjects 10 and the information processing server 20 to each other andconnecting the autonomous mobile objects 10 to each other. The network30 may include a public network, such as the Internet, a telephonenetwork or a satellite communication network, and various types of LAN(Local Area Network) and WAN (Wide Area Network) including Ethernet(trademark). The network 30 may include a dedicated network, such asIP-VPN (Internet Protocol-Virtual Private Network). The network 30 mayinclude a radio communication network, such as Wi-Fi (trademark) orBluetooth (Trademark).

The exemplary system configuration of the embodiment of the disclosurehas been described. The above-described configuration described usingFIG. 7 is an example only, and the configuration of the informationprocessing system according to the embodiment of the disclosure is notlimited to the example. For example, the autonomous mobile object 10 mayfurther communicate information with various external devices inaddition to the information processing server 20. The aforementionedexternal devices may, for example, include a server that transmits theweather and news and other service information, various informationprocessing terminal devices that the user owns, and home electricappliances. The system configuration according to the embodiment of thedisclosure can be flexibly modified according to the specification andoperation.

1.4. Exemplary Functional Configuration of Autonomous Mobile Object 10

A functional exemplary configuration of the autonomous mobile object 10according to the embodiment of the disclosure will be described. FIG. 8is a diagram illustrating an exemplary functional configuration of theautonomous mobile object 10 according to the embodiment of thedisclosure. Referring to FIG. 8, the autonomous mobile object 10according to the embodiment of the disclosure includes an input unit110, a recognition unit 120, a learning unit 130, a behavior planner140, an operation controller 150, a driver 160, an output unit 170, anda server communication unit 180.

Input Unit 110

The input unit 110 has a function of collecting various types ofinformation relating to the user and surrounding environment. The inputunit 110, for example, collects voices of the user, environmental soundoccurring around, image information relating to the user and thesurrounding environment, and various types of sensor information. Forthis reason, the input unit 110 includes each type of sensor illustratedin FIG. 1.

Recognition Unit 120

The recognition unit 120 has a function of, based on various types ofinformation collected by the input unit 110, performing various types ofrecognition relating to the user, the surrounding environment, and thecondition of the autonomous mobile object 10. In an example, therecognition unit 120 may identify a person, recognize the expression andgaze, recognize an object, recognize color, recognize a shape, recognizea marker, recognize an obstacle, recognize a step, or recognizebrightness.

The recognition unit 120 recognizes feelings relating to the voice ofthe user, understands words, and localizes a sound source. Therecognition unit 120 is capable of recognizing contact by the user, thesurrounding temperature, presence of an animal object, the posture ofthe autonomous mobile object 10, etc.

The recognition unit 120 has a function of, based on the aforementionedinformation that is recognized, estimating the surrounding environmentand circumstances in which the autonomous mobile object 10 is. Therecognition unit 120 may comprehensively estimate the circumstancesusing environmental knowledge that is stored in advance.

Learning Unit 130

The learning unit 130 has a function of learning the environment(circumstances), behavior and the effect of the behavior on theenvironment. The learning unit 130 realizes the aforementioned learningusing a machine learning algorithm, such as deep learning. The learningalgorithm that is employed by the learning unit 130 is not limited tothe example described above, and the learning algorithm may be designedappropriately.

Behavior Planner 140

The behavior planner 140 has a function of planning a behavior to betaken by the autonomous mobile object 10 based on the circumstances thatare estimated by the recognition unit 120 and the knowledge that islearned by the learning unit 130. Details of the function of thebehavior planner 140 according to the embodiment of the disclosure willbe described separately.

Operation Controller 150

The operation controller 150 has a function of controlling operations ofthe driver 160 and the output unit 170 based on the behavior plan madeby the behavior planner 140. For example, based on the aforementionedbehavior plan, the operation controller 150 performs control on rotationof the actuators 570, control on display by the displays 510, andcontrol on audio output a speaker. Details of function of the operationcontroller 150 according to the embodiment of the disclosure will bedescribed in detail separately.

Driver 160

The driver 160 has a function of bending and stretching a plurality ofjoint parts of the autonomous mobile object 10 according to control bythe operation controller 150. More specifically, according to thecontrol by the operation controller 150, the driver 160 drives theactuator 570 that each joint part has.

Output Unit 170

The output unit 170 has a function of, according to control performed bythe operation controller 150, outputting visual information and audioinformation. Thus, the output unit 170 includes the displays 510 and aspeaker.

Server Communication Unit 180

The server communication unit 180 has a function of communicatinginformation with the information processing server 20 or anotherautonomous mobile object 10. For example, the server communication unit180 transmits information on the circumstances that are recognized bythe recognition unit 120, etc., to the information processing server 20.For example, the server communication unit 180 receives control sequencedata relating to the recommended behavior and the recommended behaviorfrom the information processing server 20.

The functional configuration of the autonomous mobile object 10according to the embodiment of the disclosure has been described above.Note that the configuration described above using FIG. 8 is an exampleonly and the functional configuration of the autonomous mobile object 10according to the embodiment of the disclosure is not limited to theexample. The functional configuration of the autonomous mobile object 10according to the embodiment of the disclosure can be flexibly modifiedaccording to the specification and operation.

1.5. Exemplary Functional Configuration of Information Processing Server20

An exemplary functional configuration of the information processingserver 20 according to the embodiment of the disclosure will bedescribed. FIG. 9 is a diagram illustrating the exemplary functionalconfiguration of the information processing server 20 according to theembodiment of the disclosure. Referring to FIG. 9, the informationprocessing server 20 exemplary functional configuration of theinformation processing server 20 according to the embodiment of thedisclosure includes a learning unit 210, a behavior recommender 220, ananalyzer 230, a storage 240, and a terminal device communication unit250.

Learning Unit 210

The learning unit 130 has a function of learning the environment(circumstances), a behavior, and the effect of the behavior on theenvironment. The learning unit 210 is characterized in performinglearning based on a log of behaviors collected from the autonomousmobile objects 10. In other words, the learning unit 210 can be referredto as collective intelligence that is common among the autonomous mobileobjects 10.

Behavior Recommender 220

The behavior recommender 220 has a function of determining a recommendedbehavior that is recommended to the autonomous mobile object 10 based oninformation on the circumstances estimation that is received from theautonomous mobile object 10 and the knowledge that the learning unit 210has and that serves as the collective intelligence. The behaviorrecommender 220 has a characteristic in transmitting, together with therecommended command, the control sequence data for causing theautonomous mobile object 10 to implement the recommended behavior to theautonomous mobile object via the terminal device communication unit 250.

The aforementioned sequence data is information containing a controlsignal relating to changes in rotational position of the joint parts ofthe autonomous mobile object 10 over time, eyeball expression, and audiooutput. In other words, the control sequence data is also referred to assetting data for causing the autonomous mobile object 10 to implement agiven operation (action).

The above-described function that the behavior recommender 220 accordingto the embodiment of the disclosure has makes it possible to add newactions executable by the autonomous mobile object 10 as necessary andthus keep engaging the interest of the user in the autonomous mobileobject 10.

Analyzer 230

The analyzer 230 has a function of analyzing various types of analysisbased on information received from the autonomous mobile object 10. Theanalyzer 230 is capable of analyzing the condition of the actuator 570based on the behavior log and the operational circumstances that arereceived from the autonomous mobile object 10. The analyzer 230 iscapable or analyzing the interest (love) of the user in (for) theautonomous mobile object 10 based on the information on contact by orresponse of the user that is received from the autonomous mobile object10.

Storage 240

The storage 240 has a function of accumulating information that eachcomponent of the information processing server 20 uses. The storage 240,for example, stores the control sequence data that is received from theautonomous mobile object 10 in association with the circumstances andthe response of the user. The storage 240 stores information that isused for analysis by the analyzer 230 and the result of analysis.

Terminal Device Communication Unit 250

The terminal device communication unit 250 has a function ofcommunicating information with the autonomous mobile objects 10. Theterminal device communication unit 250, for example, receivesinformation on circumstances estimation from the autonomous mobileobject 10. The terminal device communication unit 250, for example,transmits information on the recommended behavior that is determined bythe behavior recommender 220 and the control sequence data to theautonomous mobile object 10.

The exemplary functional configuration of the information processingserver 20 according to the embodiment of the disclosure has beendescribed. Note that the configuration described above using FIG. 9 isan example only and the functional configuration of the informationprocessing server 20 according to the embodiment of the disclosure isnot limited to the example. The information processing server 20, forexample, may have a function of providing various user interfaces to bedescribed below to the user. The various functions of the informationprocessing server 20 can be distributed to multiple devices andimplemented. The functional configuration of the information processingserver 20 according to the embodiment of the disclosure can be flexiblymodified according to the modification and operation.

2. First Embodiment 2.1. Overview

The first embodiment of the disclosure will be described. As describedabove, the autonomous mobile object 10 according to the embodiment ofthe disclosure includes the displays 510 that express operations of thejoint parts and eyeballs and thus is able to execute various operations(actions).

On the other hand, in order to execute an action, the autonomous mobileobject 10 is required to have control sequence data corresponding to theaction. For this reason, when the autonomous mobile object 10 has nomechanism to add new control sequence data, the autonomous mobile object10 executes only actions that are set on shipping of goods.

In this case, the interest of the user in the actions that are executedrepeatedly is lost and this may cause lowering of the love for theautonomous mobile object 10. Depending on the user, a demand for havingthe autonomous mobile object 10 know and implement unique actions astraining a real dog is assumed.

The autonomous mobile object 10 according to the embodiment is inspiredby focusing on the above-described aspect and the user is able to easilyhave the autonomous mobile object 10 know a new action. For this reason,the autonomous mobile object 10 according to the embodiment has acharacteristic in, based on a teaching operation, generating controlsequence data for implementing an autonomous operation corresponding tothe teaching operation. The autonomous mobile object 10 according to theembodiment has a characteristic in executing an autonomous operationcorresponding to control sequence data based on a behavior plan that isdetermined by circumstances estimation.

According to the aforementioned characteristic of the autonomous mobileobject 10 according to the embodiment, the user is able to easily havethe autonomous mobile object 10 know a new action and therefore aneffect that the degree of love of the user is kept high is expected.

2.2. Generation of Control Sequence Data

Generation of control sequence data according to the embodiment will bedescribed in detail. The operation controller 150 of the autonomousmobile object 10 according to the embodiment has a function of, based ona teaching motion, generating control sequence data for implementing anautonomous operation corresponding to the teaching motion.

More specifically, the operation controller 150 according to theembodiment may generate, based on a teaching operation, control sequencedata containing at least information on changes in rotational positionsof the joint parts over time.

For example, using a dedicated user interface, the user is able to teacha new action to the autonomous mobile object 10, that is, have theautonomous mobile object 10 to know the new action. FIG. 10 is a diagramfor describing teaching an action using a user interface according tothe embodiment.

FIG. 10 illustrates a user interface UI1 for the user to teach a newaction to the autonomous mobile object 10. The user, for example, isable to access the user interface UI1 via a computer or a smartphone.

Referring to FIG. 10, the user interface UI1 includes, for example, twoareas R1 and R2. The area R1 may be an area for the user to setoperations of the joint parts of the autonomous mobile object 10. Byspecifying changes in rotational position of each joint part (direction,magnitude, and speed of bending and stretching) chronologically in thearea R1, the user is able to teach a new action to the autonomous mobileobject 10. The user may, for example, specify angles of rotation androtational speeds of the joint parts using a keyboard or mouseoperations.

The area R2 is an area where a preview of the teaching operation that isspecified by the user in the area R1 is displayed. For example, in thearea R2, an avatar of the autonomous mobile object 10 that reproducesthe operations of the joints that are specified by the user in the areaR1 may be displayed. In the area R2, in addition to the bird's-eye viewillustrated in the drawing, an image reproducing the teaching operationfrom the front side, upper side, lateral side and rear side of theautonomous mobile object 10 may be displayed.

The user is also able to teach an action by operating the avatar of theautonomous mobile object 10 that is displayed in the area R2 using amouse or fingers. The operation controller 150 according to theembodiment is capable of, according to the aforementioned user operationexecuted in the area R2, storing the angle of rotation and therotational speed of the actuator 570 corresponding to each joint partand reflect the angle of angle of rotation and the rotational speed insetting in the area R1.

Although not illustrated in the drawings, the user is also able to, inthe user interface U1, set operations of eyeballs of the autonomousmobile object 10 to be displayed on the displays 510 and a voice thatthe speaker is caused to output.

As described above, the user interface UI1 according to the embodimentallows a user to finely and accurately teach a new action and thusenables an autonomous mobile object to execute a more accurateoperation.

The teaching operation according to the embodiment includes physicalbending and stretching operations of joint parts that is implemented bythe user. The operation controller 150 according to the embodiment iscapable of generating control sequence data according to theabove-described physical bending and stretching operations.

FIG. 11 is a diagram for describing teaching according to physicalbending and stretching operations on a joint part according to theembodiment. As illustrated in FIG. 11, the user is able to teach a newaction by physically bending and stretching a joint part of theautonomous mobile object 10, for example, with a hand UA.

First of all, by giving an utterance of, for example, “learn this”, theuser has the autonomous mobile object 10 recognize that a teachingoperation is to be performed. When the recognition unit 120 recognizesthat bending and stretching operations are to be started, the operationcontroller 150 causes the driver 160 to execute a relaxing operation ona joint part.

The aforementioned relaxing operation denotes an operation of, in orderto realize a teaching operation performed by the user, causing acondition in which the actuator 570 rotates easily with respect to aforce applied from outside. The actuator 570 according to the embodimentis configured such that the drag coefficient with respect to arotational movement is adjustable to any drag coefficient, for example,enabling realization of a condition in which the actuator 570 easilyrotates with respect to a force that is applied from outside or acondition in which the actuator 570 is unable to rotate easily. Therelaxing operation according to the embodiment allows the user to easilybend and stretch the joint part easily and excludes a possibility ofdamaging the actuator 570 by forcibly applying a force and a risk thatthe user is injured.

When the user starts the bending and stretching operations of the joint,the operation controller 150 according to the embodiment stores changesin rotational position chronologically based on the rotational positionsof the joint part that are detected by the position sensor of theactuator 570.

As described above, the function of the operation controller 150according to the embodiment enables even a user who is not familiar withtechnical knowledge to intuitively teach a new action to the autonomousmobile object 10 by directly moving a joint part.

The teaching operation according to the embedment may cover movements ofan animal object having joints. The operation controller 150 accordingto the embodiment is also capable of generating control sequence databased on movements of an animal object of which images are captured.

FIG. 12 is a diagram for describing teaching relating to movements of ananimal object of which images are captured according to the embodiment.FIG. 12 illustrates visual information V1 relating to movement of ananimal object P1 that is output by a display device and the autonomousmobile object 10 that recognizes the visual information V1 by sight.

As illustrated in FIG. 12, the operation controller 150 according to theembodiment is capable of, using the movements of the animal object ofwhich images are captured by the input unit 110 as a teaching operation,generating control sequence data corresponding to the teachingoperation.

FIG. 12 illustrates the case where the movements of the animal object P1are displayed by the display device as the visual information V1;however, the movements of the animal object according to the embodimentare not limited to the example. The movements of the animal objectaccording to the embodiment widely cover, for example, real movementsmade by a user, a pet, or another autonomous mobile object and ananimation that is displayed as visual information.

The operation controller 150 according to the embodiment may executegeneration of control sequence data based on movements of an animalobject regardless whether there is a clear instruction made by the user.In other words, even when the user makes no instruction, using movementsof the user or a pet or visual information that are displayed by thedisplay device, the operation controller 150 according to the embodimentis capable of autonomously generating control sequence data relating toa new action.

The above-described function of the operation controller 150 accordingto the embodiment enables the autonomous mobile object 10 toautonomously copy movements of the user or pet or autonomously learn adance in trend that is displayed by the display device, which makes itpossible to keep the interest of the user in the autonomous mobileobject 10 high.

First of all, the operation controller 150 estimates the positions ofjoints of an animal object of which images are captured and acquiresjoint parts corresponding to the respective joints using a relativejoint map, or the like. Furthermore, the operation controller 150calculates a magnitude of bending and stretching movements relating tothe joints of the animal object, converts the bending and stretchingmovements of the joints to rotational positions of the correspondingjoint parts, and stores the rotational positions.

The operation controller 150 according to the embodiment mayappropriately correct operations of joint parts of the animal objectaccording to the ranges of motion of joint parts. For example, theoperation controller 150 according to the embodiment is able todynamically trim part of bending and stretching movements relating tojoints of the animal object and record rotational positions ofcorresponding joint parts. More specifically, when bending andstretching movements relating to the joints of the animal object exceedranges of motion of the joint parts corresponding to the joints, theoperation controller 150 according to the embodiment may dynamicallytrims part of the bending and stretching movements exceeding the rangesof motion.

FIG. 13 is a diagram for describing trimming based on a range of motionaccording to the embodiment. FIG. 13 illustrates a bending andstretching movement of the neck of an animal object P3 that is output asvisual information V2 and the autonomous mobile object 10 thatrecognizes visual information V3 by sight.

FIG. 13 illustrates an exemplary case where a bending and stretchingmovement of the neck made by the animal object P3 exceeds a range ofmotion ROM of the actuator 570 that is arranged in a neck part of theautonomous mobile object 10. The operation controller 150 according tothe embodiment may trim the bending and stretching movement of theanimal object P3 that exceeds the range of motion ROM and generatecontrol sequence data such that the rotational position of the jointpart is within the range of motion ROM.

The above-described function of the operation controller 150 accordingto the embodiment enables dynamic generation of control sequence datasuch that the teaching operation is within the range of motion of ajoint part and thus enables the autonomous mobile object 10 to realize anatural operation without stress.

The operation controller 150 according to the embodiment, for example,is also capable of generating control sequence data containingpositional information on the autonomous mobile object 10. Specifically,the operation controller 150 according to the embodiment enablesexecution an action that specifies a position of the autonomous mobileobject 10 in a space. The above-described function of the operationcontroller 150 according to the embodiment, for example, enables theautonomous mobile object to execute an action that specifies a givenspot in the house of the user or an action that specifies a physicaldistance to the user.

The operation controller 150 according to the embodiment enablesgeneration of control sequence data containing relative positionalinformation on multiple autonomous mobile objects 10.

FIG. 14 is a diagram for describing teaching that specifies relativepositions relating to multiple autonomous mobile objects 10 according tothe embodiment. FIG. 14 represents movements of animal objects P1 and P2that are output as visual information V3 and autonomous mobile objects10 a and 10 b that recognize the visual information V3 by sight.

The movements of the animal objects P1 and P2 illustrated in FIG. 14 maybe, for example, a dance that places importance on the relativepositions. The operation controller 150 according to the embodiment is,in this case, capable of, based on the relative positions of therecognized animal objects P1 and P2, generate control sequence data inwhich the relative positions with respect to the other autonomous mobileobject 10 are recorded chronologically together with the rotationalpositions of joint parts. The operation controller 150, for example,communicates information on the self-position that is estimated by theSLAM technique with the other autonomous mobile object 10 and thus isable to execute an action based on the relative positons. The operationcontroller 150 may, for example, recognize markers that are set by theuser on the floor and thus execute an action based on the relativepositions.

As described above, the operation controller 150 according to theembodiment makes it possible to easily teach a new act on to theautonomous mobile object 10 through physical operations of and imagecapturing of joints. According to the above-described function of theoperation controller 150 according to the embodiment, an effect ofmaintaining love of the user for the autonomous mobile object 10 highwithout making the user to get tired with limited actions.

The user, for example, is also able to edit a teaching operation via auser interface. FIG. 15 is a diagram for describing editing controlsequence data according to the embodiment.

FIG. 15 illustrates a user interface U12 that is used by a user to edita teaching operation. As illustrated in FIG. 15, the user interface U12has the above-described area R1 and R2. Not as in the user interfaceUI1, information of control sequence data that is taught throughphysical operations of joint parts and image capturing may be displayedin the area R1 of the user interface UI2 in advance.

Users are able to easily edit a teaching operation by moving, copying,enlarging and reducing motion bars M1 to M3 that specifies motions ofrespective joint parts with a mouse or fingers. The user interface UI2according to the embodiment, for example, makes it possible to copy theoperation of the right front leg that is taught by a physical operationof a joint part to another leg and finely specify operation timing foreach joint and thus makes it possible to perform teaching reflecting anintension of the user.

The operation controller 150 according to the embodiment is capable ofsaving the generated control sequence data and cause circumstances thatcause an autonomous operation corresponding to the control sequence datain association with each other. The cause circumstances refers tocircumstances that can cause the autonomous mobile object 10 to executethe autonomous operation corresponding to the teaching operation. Thecause circumstances according to the embodiment include various types ofcircumstances that are recognized by the recognition unit 120 based onthe sensor information that is collected by the input unit 110.

FIGS. 16 and 17 are diagrams for describing cause circumstancesaccording to the embodiment. FIG. 16 illustrates an example where amusic that is output from a display device serves as cause circumstancesand the autonomous mobile object 10 executes an autonomous operationcorresponding to a teaching operation.

In the example illustrated in FIG. 16, the music played when theteaching operation is performed is autonomously saved as causecircumstances and, on recognizing the same music, the autonomous mobileobject 10 performs the autonomous operation corresponding to theteaching operation. In this manner, the autonomous mobile object 10according to the embodiment is capable of executing the autonomousoperation corresponding to the teaching operation based on various typesof cause circumstances.

FIG. 17 illustrates an exemplary case where recognition of the userserves as cause circumstances and the autonomous mobile object 10executes an autonomous operation corresponding to a teaching operation.The cause circumstances according to the embodiment may be specified bythe user. The user, for example, is able to set the case where a user U1is recognized on the birthday of the user U1 as cause circumstances andencourage the autonomous mobile object 10 to execute an autonomousoperation corresponding to a teaching operation only once.

As described above, the autonomous mobile object 10 according to theembodiment is able to execute an autonomous operation corresponding to ateaching operation based on cause circumstances that the autonomousmobile object 10 stores or that are specified by the user. The functionenables realization of a natural response more close to that of a livingthing and an action reflecting an intention of the user.

The autonomous mobile object 10 according to the embodiment is able totransmit the control sequence data, which is generated as describedabove, to another autonomous mobile object 10. FIG. 18 is a diagram fordescribing transmission of control sequence data between the autonomousmobile objects 10 according to the embodiment.

FIG. 18 illustrates an example where control sequence data CS that isgenerated by the autonomous mobile object 10 a is transmitted to theautonomous mobile object 10 b by radio communication. As describedabove, the autonomous mobile object 10 according to the embodiment iscapable of transmitting the generated control sequence data CS toanother autonomous mobile object 10.

The above-described function of the autonomous mobile object 10according to the embodiment enables realization of insidious spread ofan operation between the autonomous mobile objects 10 that the same userowns or between the autonomous mobile objects 10 that different usersown, which thus makes it possible to keep the interest of the users highand promote communication between the users.

2.3. Flow of Control

Details of a flow of control of the autonomous mobile object 10according to the embodiment will be described. First of all, the flow ofcontrol of the autonomous mobile object 10 relating to teaching byphysical bending and stretching operations of a joint part performed bya user will be described. FIG. 19 is a flowchart representing a flow ofcontrol of the autonomous mobile object 10 relating to teaching byphysical bending and stretching operations of a joint performed by auser according to the embodiment.

Referring to FIG. 19, first of all, based on utterances of a user thatare collected by the input unit 110, the recognition unit 120 detects arequest for starting a teaching operation (S1101).

Based on the fact that the start request is detected at step S1101, theoperation controller 150 causes the driver 160 to execute a relaxingoperation (S1102).

The operation controller 150 detects a rotational position of a jointpart that is bent and stretched by the user (S1103).

The operation controller 150 records detected rotational positionschronologically (S1104).

Based on the utterances, etc., the recognition unit 120 detects the endof the teaching operation (S1105). Note that the recognition unit 120may detect the end of the teaching operation when no operation performedby the user on the joint part has been performed for a given time orlonger.

The operation controller 150 executes trimming on the recorded data(S1106). The operation controller 150 may perform trimming on the timefrom the start of detection until performing of the actual operation onthe joint part or the time from the last operation on the joint partuntil detection of the end.

The operation controller 150 converts the recorded data into controlsequence data (S1107) and ends the process.

A flow of control of the autonomous mobile object 10 using movements ofan animal object of which images are captured as a teaching will bedescribed. FIG. 20 is a flowchart representing a flow of control of theautonomous mobile object 10 using movements of an animal object of whichimages are captured as a teaching.

Referring to FIG. 20, first of all, the input unit 110 captures imagesof movements of an animal object (S1201).

The operation controller 150 then executes estimation relating topositions of joints of the animal object (S1202).

The operation controller 150 then performs mapping between the joints ofthe animal object that are estimated at step S1202 and the joint partsof the driver 160 (S1203).

The operation controller 150 converts bending and stretching of thejoints of the animal object into rotational positions of the joint partsand records the rotational positons (S1204).

Based on ranges of motion of the joint parts, the operation controller150 performs trimming on the recorded data (S1205).

The operation controller 150 converts the recorded data into controlsequence data (S1206) and ends the process.

3. Second Embodiment 3.1. Overview

A second embodiment of the disclosure will be described below. In thefirst embodiment, a method of teaching a new action to the autonomousmobile object 10 has been described in detail. In the following secondembodiment, behavior plans relating to various operations including theaforementioned action will be mainly described.

As described above, unlike an apparatus that passively operatesaccording to an instruction from a user, the autonomous mobile object 10according to the embodiment of the disclosure performs a dynamicoperation based on the estimated circumstances. The autonomous mobileobject 10 has a characteristic in performing a comprehensive behaviorplan based on multiple sets of needs opposed to each another in additionto the estimated circumstances.

For example, self-preservation needs and self-esteem needs areexemplified as the sets of needs opposed to each another. Theself-preservation needs are needs for maintenance of continuous and safeactivities of the autonomous mobile object 10. More specifically, theself-preservation needs according to the embodiment include needs formaintenance of or replenishment for the charging power of the autonomousmobile object 10. The self-preservation needs include needs formaintenance or recovery of the function of the autonomous mobile object10.

The self-esteem needs are needs for being loved, being needed, and beinginterested in. The self-esteem needs according to the embodiment thuswidely cover needs for pleasing the user, not making the user sad, etc.,in order to achieve the above-described events.

The autonomous mobile object 10 according to the embodiment of thedisclosure has both the above-described self-preservation needs andself-esteem needs and thus is able to realize more natural and flexiblevarious behavior patterns close to those of real animals. In the secondembodiment of the disclosure, flexible behavior plans of the autonomousmobile object 10 based on the above-described sets of needs andcircumstances estimation will be described in detail with specificexamples being exemplified.

3.2. Specific Example of Behavior Plan

As described above, the autonomous mobile object 10 according to theembodiment has multiple sets of needs opposed to each other, that is,self-preservation needs and self-esteem needs. Specifically, while theautonomous mobile object 10 basically has needs for being loved by theuser and for pleasing the user, the autonomous mobile object 10simultaneously has needs for reducing power consumption, for chargingpower and a demand for not wearing the components.

The behavior planner 140 according to the embodiment may, based on thecircumstances that are estimated by the recognition unit 120, plan abehavior that satisfies at least one of self-preservation needs andself-esteem needs described above. For example, the behavior planner 140is capable of planning a behavior prioritizing any one ofself-preservation needs and self-esteem needs.

For example, the behavior planner 140 may determine prioritized needsdepending on whether the user is detected. Specifically, the behaviorplanner 140 according to the embodiment may execute planning a behaviorprioritizing self-esteem needs when the user is detected and executeplanning a behavior prioritizing self-preservation needs when the useris not detected.

FIG. 21 a diagram for describing a behavior plan based on whether theuser is detected according to the embodiment. In the upper part in FIG.21, an exemplary case where a user U2 is present in an area Z1surrounding the autonomous mobile object 10 is illustrated. The behaviorplanner 140 according to the embodiment executes planning a behaviorprioritizing self-esteem needs based on the fact that the recognitionunit 120 detects the user U2 in the surrounding area Z1. For example,the behavior planner 140 may make a behavior plan of getting close tothe user U2, taking an action against the user U2, or the like.

On the other hand, in the lower part in FIG. 21, an exemplary case wherethe user is not in the area Z1 surrounding the autonomous mobile object10 is illustrated. The behavior planner 140 according to the embodimentmay plan a behavior prioritizing self-preservation needs based on thefact that the recognition unit 120 estimates that the user is notpresent. For example, the recognition unit 120, for example, mayestimate that the user is not present when the user is not in the imagethat is captured by the input unit 110 or when no utterance of the useris detected. The recognition unit 120 is also capable of estimatingabsence of the user based on the schedule information on the user.

As described above, the self-preservation needs according to theembodiment include needs relating to the charging power of theautonomous mobile object 10. When the user is not detected, the behaviorplanner 140 may plan a behavior prioritizing maintenance of orreplenishment of charging power. In the example illustrated in the lowerpart in FIG. 21, the behavior planner 140 plans replenishment for thecharging power and the operation controller 150 connects the autonomousmobile object 10 to a charger 50 based on the planning.

When power charge is not necessary, the behavior planner 140 accordingto the embodiment may execute planning various behaviors to reduce powerconsumption. FIG. 22 is a diagram for describing planning a behaviorprioritizing reduction of power consumption according to the embodiment.

For example, in order to reduce power consumption, the behavior planner140 according to the embodiment may execute planning to stop outputtingvisual expression relating to eyeball operations made by the displays510. Similarly, the behavior planner 140 is able to make a plan to stopoutput of sound by a speaker and data collection using various sensors.

The behavior planner 140 may try to reduce power consumption by delayingoperations of joint parts or planning a behavior of laying down on site.Furthermore, the behavior planner 140 may plan a behavior of turning offthe processor and the power.

The self-preservation needs according to the embodiment include needsrelating to maintenance of the function of the autonomous mobile object10. For this reason, a behavior prioritizing maintenance and recovery ofthe function of the autonomous mobile object 10 according to theembodiment may be planned.

For example, when a failure is detected in an operation of the actuator570, the behavior planner 140 may plan a behavior causing thecorresponding actuator 570 to operate as little as possible in order notto deteriorate the failure. For example, the behavior planner 140 mayprioritize maintaining the function and plan execution of calibrationrelating to the display 510 and various sensors.

When the degree of self-preservation needs described above issignificantly large, the behavior planner 140 may plan a behaviorprioritizing self-preservation needs. For example, when the chargingpower is about to deplete or when the damage of the actuator 570 isserious, the behavior planner 140 plans a behavior prioritizingself-preservation needs even when the user is present.

On the other hand, when the degree of self-preservation needs is under athreshold, the behavior planner 140 enables the autonomous mobile object10 to implement various operations for meeting expectation of the userby planning a behavior prioritizing self-esteem needs.

As described above, the behavior planner 140 according to the embodimentenables implementation of complicated and flexible behavior patternsclose to those of real animals by controlling the priorities ofself-preservation needs and self-esteem needs according to thecircumstances.

The behavior planner 140 according to the embodiment may, even when anyone of self-preservation needs and self-esteem needs is prioritized,plan a behavior that can satisfy both self-preservation needs andself-esteem needs at a time. For example, there is a case where, whenthe user is detected in the field of view or when being called by theuser, it is possible to reduce power consumption by taking a behaviorcorresponding to the condition of the user without rushing over to theuser.

FIG. 23 is a diagram illustrating an exemplary behavior plan thatsatisfies both self-preservation needs and self-esteem needs accordingto the embodiment. FIG. 23 illustrates an example of planning a behavioron a call by the user.

First of all, when the recognition unit 120 detects an utterance of theuser, the behavior planner 140 plans a behavior of turning the gaze thatis displayed on the displays 510 toward the user. The above-describedfunction of the behavior planner 140 makes it possible to, by first ofall controlling only visual information relating to eyeball operations,realize a quick response and prevent the actuator 570 to operateunnecessarily.

When it is recognized that the user calls the autonomous mobile object10 or the gaze of the user is toward the autonomous mobile object 10,the recognition unit 120 turns the head part and the torso part towardthe user sequentially after the gaze. The behavior planner 140 is ableto implement more natural operations of the autonomous mobile object 10by, while turning the irises back to the centers of the displays 510with the eyes setting on the user, turning the head part toward the usersimultaneously.

In the same manner, the behavior planner is able to, by planning abehavior such that an angular difference with respect to the torso partgradually reduces with the head turned to the user, realize naturalmotions and prevent the power consumption from increasing due to asudden operation.

The behavior planner 140 according to the embodiment may plan a behaviorbased on a distance between the user and the autonomous mobile object10. FIG. 24 is a diagram for explaining a behavior plan based on adistance between the user and the autonomous mobile object 10 accordingto the embodiment.

FIG. 24 illustrates an exemplary behavior that is planned according to adistance to the user U2. Specifically, as illustrated in the upper partin FIG. 24, when the distance d between the user U2 and the autonomousmobile object 10 is long, the behavior planner 140 first of all may plana behavior of turning only the gaze toward the user.

As illustrated in the middle part in FIG. 24, when the distance dbetween the user U2 and the autonomous mobile object 10 is intermediate,the behavior planner 140 plans a behavior of turning the head parttoward the user after the gaze.

As illustrated in the lower part in FIG. 24, when the distance d betweenthe user U2 and the autonomous mobile object 10 is short, the behaviorplanner 140 plans a behavior of turning the torso part toward the userafter the gaze and the head part.

As described above, the behavior planner 140 according to the embodimentis able to plan a flexible behavior based on the distance between theuser and the autonomous mobile object 10. The above-described functionof the behavior planner 140 enables a behavior of, while reducing powerconsumption by first of all shifting only the gaze, causing the actuator570 to operate according to the degree of proximity of the userafterwards, which thus makes it possible to efficiently reduce powerconsumption while showing a definite response to the user.

The behavior planner 140 according to the embodiment may plan thebehavior described above based on the strength of the demand of theuser. When the user keeps calling the autonomous mobile object 10 orgets close to the autonomous mobile object 10 after the gaze is turned,the behavior planner 140 may plan a behavior of turning the head partand the torso part.

The behavior planner 140 according to the embodiment may plan a behaviorbased on an emotional distance to the user other than the physicaldistance to the user. The behavior planner 140, for example, is able toplan a behavior of, while only turning the gaze when called from adistance soon after the user uses the autonomous mobile object 10,rushing to the user even when called from a distance after the use ofthe autonomous mobile object 10 is kept sufficiently.

The behavior planner 140 according to the embodiment is capable ofplanning, in addition to the above-described behaviors, variousbehaviors based on changes in the condition of surrounding environmentand the condition of the user. For example, the behavior planner 140according to the embodiment is able to plan a flexible behavior based ondetected changes in illuminance environment.

FIGS. 25 to 27 are diagrams illustrating examples of planning behaviorsbased on changes in environmental condition according to the embodiment.FIG. 25 illustrates the autonomous mobile object 10 that is stored bythe user U2 in a box or a carry bag. In this case, the behavior planner140 according to the embodiment may plan a behavior of autonomouslyturning off the power based on the fact that the recognition unit 120detects that the user U2 houses the autonomous mobile object 10.

The above-described function of the behavior planner 140 makes itpossible to satisfy self-preservation needs relating to charging powerand maintenance of the function and simultaneously satisfy self-esteemneeds by executing a behavior considering the intention of the user.

The recognition unit 120 is capable of recognizing an act of the user U2to house the autonomous mobile object 10, for example, based on a dropin illuminance and restriction of an obstacle on operations of theactuator 570.

For example, when the user U2 houses the autonomous mobile object 10with the head part being outside in a tote bag, or the like, thebehavior planner 140 may plan a behavior such that the actuators of thetorso part and leg parts do no operate while the output of visualinformation relating to the eyeball operations performed by the displays510 and operations of the head part, the ears and the mouse aremaintained.

FIGS. 26 and 27 illustrate behaviors of the autonomous mobile object 10taken when the user puts a blanket, or the like, on the autonomousmobile object 10. In this case, the behavior planner 140 according tothe embodiment may plan a behavior satisfying at least one ofself-preservation needs and self-esteem needs based on the face that asudden change in illuminance environment is detected. In that case, thebehavior planner 140 may determine priorities relating toself-preservation needs and self-esteem needs based on the surroundingenvironment and the condition of the user.

For example, in the example illustrated in FIG. 26, the behavior planner140 plans a behavior of escaping from the blanket based on the fact thatthe time is early for the user to sleep and that laughing voice of theuser is detected. The behavior planner 140 may plan a behavior of theautonomous mobile object 10 to move toward a brighter site.

As described above, the behavior planner 140 according to the embodimentenables, when mischief of the user is estimated from the conditionincluding the response of the user and the surrounding environment,implementation of a behavior along expectation of the user by planning aresponse like that of a real dog.

In the example illustrated in FIG. 27, based on the fact that it is timefor the user to sleep and the user wears a night wear, the behaviorplanner 140 plans a behavior of sleeping with the user. Specifically,the behavior planner 140 may plan a behavior of turning off the powerafter the autonomous mobile object 10 is in a supine position.

As described above, when it is estimated that the user is going to sleeptogether from the state of the user and the surrounding environment, thebehavior planner 140 according to the embodiment is able to satisfy bothself-esteem needs and self-preservation needs by performing a behaviorthat meets the expectation of the user and reduces power consumption.

The behavior planner 140 according to the embodiment, for example, isalso able to plan a behavior based on a control mode of the autonomousmobile object 10. For example, a silent mode in which no sound output isperformed is taken as the aforementioned control mode.

FIG. 28 is a diagram for explaining a behavior plan based on the controlmode of the autonomous mobile object 10 according to the embodiment. Onthe left s in FIG. 28, the behavior of the autonomous mobile object 10in the case where the control mode of the autonomous mobile object 10 isa normal mode is exemplified. The behavior planner 140 according to theembodiment may plan a behavior of outputting voices and opening themouse in response to a call by the user, or the like.

On the right in FIG. 28, a behavior of the autonomous mobile object 10in the case where the control mode of the autonomous mobile object 10 isthe silent mode is exemplified. The behavior planner 140 according tothe embodiment may plan a behavior of outputting no voce and not openingthe mouse according to the silent mode.

The above-described function of the behavior planner 140 according tothe embodiment makes it possible to realize more natural operations byclosing the mouse when no voice is output and at the same timeeffectively reduce power consumption by not opening the mouse.

The behavior planner 140 may realize a response to the user by planning,instead of operations of the mouse, a change in output of visualinformation relating to the eyeball operations performed by the displays510 or a non-verbal behavior using or the tail.

As described above, the behavior planner 140 according to the embodimentis able to implement a behavior that satisfy both self-preservationneeds and self-esteem needs.

The silent mode according to the embodiment may be set by the user ormay be set as part of planning a behavior by the behavior planner 140.The behavior planner 140 is also able to plan an autonomous shift to thesilent mode, for example, when it is detected that the user is talkingwith another person.

3.3. Flow of Control

A flow of a behavior plan of the autonomous mobile object 10 accordingto the embodiment will be described in detail. FIG. 29 is a flowchartrepresenting a flow of the behavior plan according to the embodiment.

Referring to FIG. 29, first of all, the input unit 110 collects sensorinformation (S2101).

The recognition unit 120 then executes each type of recognition processbased on the sensor information that is collected at step S2101 (S2102).

The recognition unit 120 performs comprehensive circumstances estimationbased on various events that are recognized at step S2102 (S2103).

Collecting sensor information, the recognition processes, thecircumstances estimation at steps S2101 to 2103 may be continuouslyexecuted constantly.

The behavior planner 140 plans a behavior to be executed by theautonomous mobile object 10 based on the circumstances estimated at stepS2103 and self-preservation needs and self-esteem needs (S2104).

Based on the behavior plan that is determined at step S2104, theoperation controller 150 controls operations of the driver 160 and theoutput unit 170 to execute the behavior (S2105).

4. Third Embodiment 4.1. Overview

A third embodiment of the disclosure will be described. In the first andsecond embodiment, the behavior planning function and the motion controlfunction of the autonomous mobile object 10 have been mainly described.On the other hand, in a third embodiment of the disclosure, the functionof the information processing server 20 will be focused on anddescribed.

As described above, the autonomous mobile object 10 according to theembodiment of the disclosure has the circumstances estimation function,the behavior planning function and the operation control function and iscapable of taking autonomous behaviors. In other words, the autonomousmobile object 10 is an apparatus capable of operating independently. Onthe other hand, when performing operations completely independently, theautonomous mobile object 10 performs leaning based on only theoperations executed by the autonomous mobile object 10 and it isdifficult to share the result of leaning with other autonomous mobileobjects 10.

Even when the autonomous mobile object 10 performs operationsindependently, it is possible to increase actions that can be taken byteaching by the user: however, more room is discerned to collect moreactions that attract the interest of the user.

The information processing server 20 according to the embodiment isinvented by focusing on the above-described aspect and providingcollective intelligence based on a log of behaviors that are collectedfrom the multiple autonomous mobile objects 10 enables each autonomousmobile object 10 to take more appropriate behaviors.

The information processing server 20 thus includes the behaviorrecommender 220 that represents, to the autonomous mobile object 10 thatperforms a behavior plan based on circumstances estimation, arecommended behavior to be recommended to the autonomous mobile object10. The behavior recommender 220 has a characteristic in determining theaforementioned recommended behavior based on the log of behaviorscollected from the autonomous mobile objects 10 and a circumstancessummary that is received from the autonomous mobile object 10 to whichrecommendation is made (also referred to as a subject autonomous mobileobject).

The function of the information processing server 20 according to theembodiment and the effect achieved by the function will be described indetail below.

4.2. Representation of Recommended Behavior

First of all, the function of representing a recommended behaviorimplemented by the behavior recommender 220 according to the embodimentwill be described. As described above, the autonomous mobile object 10according to the embodiment is capable of independently performing abehavior plan based on circumstances estimation. Depending on thecircumstances, however, the case where reliability relating to abehavior plan is not sufficient and the case where plans tend to beuniform are assumed. They can be causes of decreases in satisfaction ofthe user with the autonomous mobile object 10 and the degree of love ofthe user for the autonomous mobile object 10.

For this reason, by representing a recommended behavior to the subjectautonomous mobile object based on the log of behaviors collected fromthe autonomous mobile objects 10, the information processing server 20according to the embodiment is able to support the subject autonomousmobile object to perform more appropriate operations.

FIG. 30 is a diagram for describing representation of a recommendedbehavior according to the embodiment. FIG. 30 illustrates the autonomousmobile object 10 that is a subject autonomous mobile object and theinformation processing server 20. Note that FIG. 30 illustrates twoautonomous mobile objects 10 for description and the two autonomousmobile objects 10 are an identical subject autonomous mobile object.

On the left in FIG. 30, the exemplary case where the autonomous mobileobject 10 that is the subject autonomous mobile object independentlyperforms a behavior plan based on circumstances estimation. Theautonomous mobile object 10 may, for example, plan an operation that isrepresented by the balloon on the left in FIG. 30.

On the other hand, the behavior recommender 220 of the informationprocessing server 20 according to the embodiment is capable ofdetermining a recommended behavior to be recommended to the subjectautonomous mobile object based on summary information (also referred toas circumstances summary) relating to circumstances estimation that isreceived from the autonomous mobile object 10 that is the subjectautonomous mobile object and representing the recommended behavior tothe subject autonomous mobile object. The recommended behavior that isrecommended by the behavior recommender 220 may be the operation that isrepresented by the balloon on the upper right in FIG. 30.

The behavior recommender 220 according to the embodiment has acharacteristic in providing the control sequence data CS forimplementing an operation corresponding to the recommended behavior tothe subject mobile object via the terminal device communication unit250.

As described above, the behavior recommender 220 according to theembodiment is capable of representing a recommended behavior and controlsequence data relating to the recommended behavior to a subjectautonomous mobile object, which enables the subject autonomous mobileobject to execute a new action to which a good response of the user isexpected.

Representation of a recommended behavior made by the behaviorrecommender 220 according to the embodiment and a flow of a behaviorplan of the autonomous mobile object 10 will be described in detail.FIG. 31 is a conceptual view to describe a behavior plan based on therecommended behavior according to the embodiment.

First of all, the recognition unit 120 of the autonomous mobile object10 executes various types of recognition and circumstances estimationbased on sensor information that is collected by the input unit 110. Therecognition unit 120 passes the result of circumstances estimation tothe behavior planner 140 and transmits a circumstances summary to thebehavior recommender 220 of the information processing server 20.

The behavior recommender 220 according to the embodiment determines arecommended behavior using the circumstances summary that is receivedfrom the recognition unit 120 and knowledge serving as collectiveintelligence relating to the autonomous mobile objects 10 of thelearning unit 210 and represents information relating to the recommendedbehavior to the behavior planner 140.

The behavior planner 140 according to the embodiment then determines abehavior to be actually executed based on multiple possible behaviorsbased on the circumstances estimation made by the recognition unit 120and the recommended behavior that is recommended by the behaviorrecommender 220. The behavior planner 140 may make a final behaviordetermination based on reliability of each of the possible behaviors. Asdescribed above, the behavior planner 140 according to the embodimentneed not necessarily employ a recommended behavior.

The operation controller 150 then controls the driver 160 and the outputunit 170 based on the behavior plan that is determined by the behaviorplanner 140 and implements an operation performed by the autonomousmobile object 10.

The behavior planner 140 associates the circumstances estimation made bythe recognition unit 120, the determined behavior plan, and a response(feedback) of the user U2 to the executed operation with one another andtransmits them to the information processing server 20.

The above-described information is stored as the behavior log in thestorage 240 of the information processing server 20 and is used forlearning by the learning unit 210. The feedback of the user may bestored in a quantified condition via analysis by the analyzer 230. Theanalyzer 230, for example, is able to quantify positiveness andnegativeness of response of the user based on the expression andutterances of the user.

As described above, the information processing system according to theembodiment is capable of effectively learning operations that attractthe interest of users more by estimating the circumstances, representinga recommended behavior, planning a behavior, controlling operations,collecting the behavior log, and repeated execution.

The behavior recommender 220 according to the embodiment, for example,may represent a recommended behavior based on the degree of love of auser for a subject autonomous mobile object. FIG. 32 is a diagram forfor describing representation of a recommended behavior based on adegree of love of the user according to the embodiment.

FIG. 32 illustrates the user U2 with lowered love for the autonomousmobile object 10 that is the subject autonomous mobile object. Thebehavior recommender 220 according to the embodiment may, based on theanalysis by the analyzer 230 indicating that the love of the user U2 islowering, represent a recommended behavior or provide the controlsequence data CS to the autonomous mobile object 10.

The analyzer 230 is able to analyze the aforementioned degree of lovebased on the feedback of the user U2 to the operation executed by theautonomous mobile object 10, the number of times the user U2 is incontact with the autonomous mobile object 10, the number of calls, thetime during which the autonomous mobile object 10 is on, etc.

Analysis on the degree of love may be executed by the autonomous mobileobject 10. In that case, based on the fact that the degree of love ofthe user is lowering, the autonomous mobile object 10 makes a requestfor a recommended behavior to the information processing server 20.Based on the request for the circumstances, the behavior recommender 220is able to represent a recommended behavior to the autonomous mobileobject 10.

The above-described functions of the information processing server 20and the autonomous mobile object 10 according to the embodiment enableefficient increase of new actions that can be executed by the autonomousmobile object 10 and enables prevention of the degree of love of theuser from lowering.

A system for the behavior recommender 220 according to the embodiment tocollect control sequence data from the autonomous mobile objects 10 willbe described. FIG. 33 is a diagram for describing collection of controlsequence data according to the embodiment.

FIG. 33 illustrates the autonomous mobile object 10 that executes anoperation and the user U2 that makes a positive feedback to theoperation. When the feedback of the user to the executed operation ispositive as described above, the autonomous mobile object 10 maytransmit a circumstances summary, control sequence data CS relating tothe executed operation, and feedback of the user as a behavior log tothe information processing server 20.

As described above, the behavior recommender 220 according to theembodiment is capable of efficiently collect, from the multipleautonomous mobile objects 10, control sequence data corresponding tooperations to which the user shows positive feedback. Theabove-described system enables provision of the control sequence datathat is collected from other autonomous mobile objects 10 to a subjectautonomous mobile object and enables the autonomous mobile objects 10 toshare the operations that are assumed to be effective to the user.

Downloading and uploading the control sequence data according to theembodiment, for example, may be performed freely by the user via aclient application. The user may restrict the range of disclosure ofcontrol sequence data, for example, to a group of friends or work place.The function enables the group to spread and share the operations lovedby the group and thus an effect of promotion of communication amongusers is also expected.

Together with the aforementioned downloading, the user is also able toset cause circumstances described in the first embodiment via the clientapplication. FIG. 34 is a diagram illustrating an exemplary userinterface of the client application according to the embodiment.

FIG. 34 illustrates a user interface UI3 that makes it possible to setcause circumstances and download control sequence data. The user may,for example, select cause circumstances and an operation correspondingto the cause circumstance from options OP1 and OP2 on the user interfaceUI3.

In the example illustrated in FIG. 34, the user is able to select causecircumstances, such as “if it rains”, “if the user goes home”, or “ifthe autonomous mobile object 10 is in a bad mood”, from options OP1. Forexample, a field for specifying detailed circumstances may be set forcause circumstances.

The user is able to specify a given operation to be associated withcause circumstances from the options OP2. The user, for example, may beable to check a preview of operations by pushing the buttons b1 to b3.By selecting a given operation while checking the preview of operationsand pushing a button b4, the user is able to download the controlsequence data on the operation associated with a cause condition to theautonomous mobile object 10.

4.3. Additional Registration in Recognition Dictionary

A function of making additional registration in a recognition dictionarythat the behavior recommender 220 according to the embodiment has willbe described. The behavior recommender 220 according to the embodimentmay have a function of, in addition to representing a recommendedbehavior to the autonomous mobile object 10, additionally registeringnew data in an object recognition dictionary or an audio recognitiondictionary of the autonomous mobile object 10.

FIG. 35 is a diagram for describing additional registration in theobject recognition dictionary according to the embodiment. On the leftin FIG. 35, the user U2 who causes the autonomous mobile object 10 tonewly learn object recognition relating to apples is illustrated.

When an audio recognition result and image data relating to “apple” areregistered in association with each other in an object recognitiondictionary 122 a of the autonomous mobile object 10 because of theabove-described act, the behavior recommender 220 according to theembodiment may collect data that is newly registered in the objectrecognition dictionary 122 a and additionally register the data in anobject recognition dictionary 122 b of the autonomous mobile object 10 bthat is a subject autonomous mobile object.

The above-described function of the behavior recommender 220 accordingto the embodiment makes it possible to efficiently enhance the contentof the object recognition dictionary 122 of the autonomous mobile object10 and improve generalization relating to object recognition.

FIG. 36 is a diagram for describing additional registration in the audiorecognition dictionary according to the embodiment. On the left in FIG.36, the autonomous mobile object 10 that fails in audio recognition ofan utterance made by the user U2 is illustrated. The behaviorrecommender 220 according to the embodiment collects a recognitionfailure log relating to utterances made by the user U2 from theautonomous mobile object 10. The recognition failure log contains audiodata including utterances of the user U2.

Subsequently, the behavior recommender 220 causes a plurality ofrecognition engines 60 a to 60 c to recognize the sound of utterance ofthe user contained in the collected recognition failure log, therebyacquiring recognition results. When likely data is obtained from theacquired recognition results, the behavior recommender 220 mayadditionally register the data in an audio recognition dictionary 124 ofthe autonomous mobile object 10.

The above-described function of the behavior recommender 220 accordingto the embodiment makes it possible to efficiently enhance the contentof the audio recognition dictionary 124 of the autonomous mobile object10 and improve generalization relating to object recognition.

4.4. Recommendation of Maintenance

A function of recommending maintenance that the behavior recommender 220according to the embodiment has will be described. The behaviorrecommender 220 according to the embodiment may have a recommendationfunction relating to maintenance of the autonomous mobile object 10 inaddition to representation of a recommended behavior to the autonomousmobile object 10.

FIG. 37 is a diagram for describing a function of recommendingmaintenance according to the embodiment. On the left in FIG. 37, theautonomous mobile object 10 in which a failure occurs in the actuator570 of the right front leg is illustrated. Based on a result of analysisby the analyzer 230 on the operational status relating to the autonomousmobile object 10, the behavior recommender 220 according to theembodiment is able to transmit a notification recommending maintenanceto the user.

The analyzer 230 is capable of, based on information relating to theoperational status of a component, such as the actuator 570, that isreceived from the autonomous mobile object 10, detecting or expectingdeterioration or a failure of the component. For example, in addition tothe total number of times of operation and the total operational time, abehavior failure log relating to the component is taken as theaforementioned information relating to the operational status. Theaforementioned behavior failure includes a log of error notificationsthat are output when the component does not operate according to controlby the operation controller 150.

When a failure of the component is estimated from the behavior failurelog, the behavior recommender 220 according to the embodiment maytransmit a notification recommending maintenance of the component to theinformation processing terminal device 40 that the user owns. Thebehavior recommender 220 may transmit the aforementioned notificationvia a client application like that described above.

When a failure of the component is detected or expected, the behaviorrecommender 220 according to the embodiment may be able to automaticallyorder the component.

The behavior recommender 220 and the analyzer 230 according to theembodiment enable early detection of a failure of a component of theautonomous mobile object 10 and utilization of the autonomous mobileobject 10 in a safe condition for a long term.

4.5. Flow of Control

A flow of representation of a recommended behavior made by theinformation processing server 20 according to the embodiment will bedescribed in detail. FIG. 38 is a flowchart representing a flow ofrepresentation of a recommended behavior made by the informationprocessing server 20 according to the embodiment.

Referring to FIG. 38, first of all, the terminal device communicationunit 250 receives a circumstances summary from a subject autonomousmobile object (S3101).

Based on the circumstances summary that is received at step S3101 andknowledge serving as collective intelligence of the learning unit 210,the behavior recommender 220 determines a recommended behavior (S3102).

The behavior recommender 220 acquires control sequence datacorresponding to the recommended behavior that is determined at stepS3102 from the storage 240 (S3103).

The behavior recommender 220 subsequently transmits information relatingto the recommended behavior that is determined at step S3102 and thecontrol sequence data that is acquired at step S3103 to the subjectautonomous mobile object via the terminal device communication unit 250(S3104).

5. Exemplary Hardware Configuration

An exemplary hardware configuration of the information processing server20 according to an embodiment of the disclosure will be described. FIG.39 is a block diagram illustrating an exemplary hardware configurationof the information processing server 20 according to the embodiment ofthe disclosure. Referring to FIG. 39, the information processing server20, for example, includes a CPU 871, a ROM 872, a RAM 873, a host bus874, a bridge 875, an external bus 876, an interface 877, an inputdevice 878, an output device 879, a storage 880, a drive 881, aconnection port 882, and a communication device 883. The hardwareconfiguration illustrated herein is an example and part of thecomponents may be omitted. Components other than the componentsillustrated herein may be further included.

CPU 871

The CPU 871, for example, functions as an arithmetic processor or acontrol device and controls all or part of operations of the componentsbased on various programs that are recorded in the ROM 872, the RAM 873,the storage 880, or the removable recording medium 901.

ROM 872, RAM 873

The ROM 872 is a unit that stores a program to be loaded in the CPU 871and data used for computation, etc. In the RAM 873, for example, aprogram to be loaded in the CPU 871 and various parameters that vary asappropriate when the program is executed, etc., are temporarily orpermanently stored.

Host Bus 874, Bridge 75, External Bus 876 and Interface 877

The CPU 871, the ROM 872 and the RAM 873 are, for example, connected toone another via the host bus 874 capable of high-rate data transmission.On the other hand, the host bus 874, for example, is connected to theexternal bus 876 in which the data transmission rate is relatively lowvia the bridge 875. The external bus 876 is connected to variouscomponents via the interface 877.

Input Device 878

For example, a mouse, a keyboard, a touch panel, a button, a switch anda lever, etc., are used for the input device 878. A remote controllercapable of transmitting a control signal using infrared rays or otherradio waves may be used as the input device 878. The input device 878includes an audio input device, such as a microphone.

Output Device 879

The output device 879 is, for example, a device capable of visually orauditorily notifying a user of acquired information, such as a displaydevice like a CRT (Cathode Ray Tube), a LCD, or an organic EL, an audiooutput device like a speaker or headphones, a printer, a mobile phone,or a facsimile machine. The output device 879 according to thedisclosure includes various vibration devices capable of outputtingtactile stimulation.

Storage 880

The storage 880 is a device for storing various types of data. Forexample, a magnetic storage device, such as a hard disk drive (HDD), asemiconductor storage device, an optical storage device, or anmagneto-optical device may be used as the storage 880.

Drive 881

The drive 881, for example, is a device that reads information recordedin the removable recording medium 901, such as a magnetic disk, anoptical disk, a magneto-optical disk, or a semiconductor memory, orwrites information in the removable recording medium 901.

Removable Recording Medium 901

The removable recording medium 901 is, for example, a DVD medium, aBlu-ray (trademark) medium, a HD DVD medium, or various types ofsemiconductor storage media. The removable recording medium 901, forexample, may be an IC card, an electric device, or the like, on which anon-contact IC chip is mounted.

Connection Port 882

The connection port 882, for example, is a port for connecting theexternal connection device 902, such as a USB (Universal Serial Bus)port, an IEEE1394 port, a SCSI (Small Computer System Interface), aRS-232C port or an optical audio terminal.

External Connection Device 902

The external connection device 902 is, for example, a printer, aportable music player, a digital camera, a digital video camera or an ICrecorder.

Communication Device 883

The communication device 883 is a communication device for connecting toa network and is, for example, a communication card for a wired orwireless LAN, Bluetooth (trademark) or WUSB (Wireless USB), a router foroptical communication, a router for ADSL (Asymmetric Digital SubscriberLine), or a modem for various types of communication.

6. Summary

As described above, the autonomous mobile object 10 according to theembodiment of the disclosure includes the behavior planner 140 thatplans a behavior based on circumstances estimation. The behavior planner140 has a characteristic in determining a behavior to be executed by theautonomous mobile object based on the estimated circumstances andmultiple sets of needs opposed to each other. The configuration enablesimplementation of a natural and flexible behavior plan of the autonomousmobile object.

The preferable embodiments of the disclosure have been described indetail with reference to the accompanying drawings; however, thetechnical scope of the disclosure is not limited to the examples. It isobvious that those with general knowledge in the technical field of thedisclosure can reach various exemplary modifications or exemplarycorrections within the scope of technical idea described in the claimsand it is understood that they naturally belong to the technical scopeof the disclosure.

The effects disclosed herein are explanatory and exemplary only and thusare not definitive. In other words, the technique according to thedisclosure can achieve, together with the above-described effects orinstead of the above-described effects, other effects obvious to thoseskilled in the art from the description herein.

Each step according to the processes performed by the autonomous mobileobject 10 and the information processing server 20 according to thedisclosure need not necessarily be processed chronologically along theorder illustrated in the flowchart. For example, each step according tothe processes of the autonomous mobile object 10 and the informationprocessing server 20 may be processed in an order different from thatillustrated in the flowchart or may be processed in parallel.

The following configuration also belongs to the technical scope of thedisclosure.

(1)

An information processing apparatus comprising:

a behavior planner configured to plan a behavior of an autonomous mobileobject based on estimation of circumstances,

wherein the behavior planner is configured to, based on thecircumstances that are estimated and multiple sets of needs that areopposed to each other, determine the behavior to be executed by theautonomous mobile object.

(2)

The information processing apparatus according to (1), wherein the setsof needs opposed to each other include self-preservation needs andself-esteem needs.

(3)

The information processing apparatus according to (2), wherein thebehavior planner is configured to, based on the estimated circumstances,plan the behavior satisfying at least one of the self-preservation needsand the self-esteem needs.

(4)

The information processing apparatus according to (2) or (3), whereinthe behavior planner is configured to, in circumstances where no user isdetected, plan the behavior prioritizing the self-preservation needs.

(5)

The information processing apparatus according to any one of (2) to (4),wherein.

the self-preservation needs include needs relating to charging power ofthe autonomous mobile object, and

the behavior planner is configured to plan the behavior prioritizingmaintenance of or replenishment for the charging power.

(6)

The information processing apparatus according to any one of (2) to (5),wherein

the self-preservation needs include needs relating to maintenance of afunction of the autonomous mobile object, and

the behavior planner is configured to plan the behavior prioritizingmaintenance or recovery of the function of the autonomous mobile object.

(1)

The information processing apparatus according to any one of (2) to (6),wherein the behavior planner is configured to, when theself-preservation needs are under a threshold, plan the behaviorprioritizing the self-esteem needs.

(8)

The information processing apparatus according to any one of (2) to (7),wherein

the autonomous mobile object includes a structure corresponding to atleast eyeballs and a head part, and

the behavior planner is configured to plan the behavior of turning gazetoward a user and then turning the head part toward the user.

(9)

The information processing apparatus according to (8), wherein

the autonomous mobile object further includes a structure correspondingto a torso part, and

the behavior planner is configured to plan the behavior of turning thetorso part toward the user after turning the head part toward the user.

(10)

The information processing apparatus according to any one of (1) to (9),wherein the behavior planner is configured to plan the behavior based ona distance between the autonomous mobile object and the user.

(11)

The information processing apparatus according to any one of (1) to(10), wherein the behavior planner is configured to plan the behaviorbased on strength of a strength of a demand of the user for theautonomous mobile object.

(12)

The information processing apparatus according to any one of (1) to(11), wherein the behavior planner is configured to plan the behaviorbased on a control mode of the autonomous mobile object.

(13)

The information processing apparatus according to (12), wherein

the autonomous mobile object includes a structure corresponding to amouth, and

the behavior planner is configured to, when the control mode of theautonomous mobile object is a silent mode, plan the behavior such thatthe structure corresponding to the mouth does not open.

(14)

The information processing apparatus according to any one of (1) to(13), wherein the behavior planner is configured to, based on changes inilluminance, plan the behavior satisfying at least one ofself-preservation needs and self-esteem needs.

(15)

The information processing apparatus according to (14), wherein thebehavior planner is configured to, based on a surrounding environment ora condition of the user, determine a priority relating to theself-preservation needs or the self-esteem needs.

(16)

The information processing apparatus according to any one of (1) to(15), wherein the behavior planner is configured to, when it is detectedthat the user houses the autonomous mobile object, plan the behaviorsatisfying both self-preservation needs and self-esteem needs.

(17)

The information processing apparatus according to any one of (1) to(16), further comprising an operation controller configured to controloperations of the autonomous mobile object based on the behavior plannedby the behavior planner.

(18)

The information processing apparatus according to any one of (1) to(17), further comprising independent two display devices correspondingto eyeballs.

(19)

An information processing method comprising:

by a processor, planning a behavior of an autonomous mobile object basedon estimation of circumstances, wherein

the planning includes, based on the circumstances that are estimated andmultiple sets of needs that are opposed to each other, determining thebehavior to be executed by the autonomous mobile object.

(20)

A program for causing a computer to function as an informationprocessing apparatus comprising a behavior planner configured to plan abehavior of an autonomous mobile object based on estimation ofcircumstances,

wherein the behavior planner is configured to, based on thecircumstances that are estimated and multiple sets of needs that areopposed to each other, determine the behavior to be executed by theautonomous mobile object.

REFERENCE SIGNS LIST

-   10 Autonomous Mobile Object-   110 Input Unit-   120 Recognition Unit-   130 Learning Unit-   140 Behavior Planner-   150 Operation Controller-   160 Driver-   170 Output Unit-   510 Display-   570 Actuator-   20 Information Processing Server-   210 Learning Unit-   220 Behavior Recommender-   230 Analyzer-   240 Storage

1. An information processing apparatus comprising: a behavior planner configured to plan a behavior of an autonomous mobile object based on estimation of circumstances, wherein the behavior planner is configured to, based on the circumstances that are estimated and multiple sets of needs that are opposed to each other, determine the behavior to be executed by the autonomous mobile object.
 2. The information processing apparatus according to claim 1, wherein the sets of needs opposed to each other include self-preservation needs and self-esteem needs.
 3. The information processing apparatus according to claim 2, wherein the behavior planner is configured to, based on the estimated circumstances, plan the behavior satisfying at least one of the self-preservation needs and the self-esteem needs.
 4. The information processing apparatus according to claim 2, wherein the behavior planner is configured to, in circumstances where no user is detected, plan the behavior prioritizing the self-preservation needs.
 5. The information processing apparatus according to claim 2, wherein the self-preservation needs include needs relating to charging power of the autonomous mobile object, and the behavior planner is configured to plan the behavior prioritizing maintenance of or replenishment for the charging power.
 6. The information processing apparatus according to claim 2, wherein the self-preservation needs include needs relating to maintenance of a function of the autonomous mobile object, and the behavior planner is configured to plan the behavior prioritizing maintenance or recovery of the function of the autonomous mobile object.
 7. The information processing apparatus according to claim 2, wherein the behavior planner is configured to, when the self-preservation needs are under a threshold, plan the behavior prioritizing the self-esteem needs.
 8. The information processing apparatus according to claim 1, wherein the autonomous mobile object includes a structure corresponding to at least eyeballs and a head part, and the behavior planner is configured to plan the behavior of turning gaze toward a user and then turning the head part toward the user.
 9. The information processing apparatus according to claim 8, wherein the autonomous mobile object further includes a structure corresponding to a torso part, and the behavior planner is configured to plan the behavior of turning the torso part toward the user after turning the head part toward the user.
 10. The information processing apparatus according to claim 1, wherein the behavior planner is configured to plan the behavior based on a distance between the autonomous mobile object and the user.
 11. The information processing apparatus according to claim 1, wherein the behavior planner is configured to plan the behavior based on strength of a strength of a demand of the user for the autonomous mobile object.
 12. The information processing apparatus according to claim 1, wherein the behavior planner is configured to plan the behavior based on a control mode of the autonomous mobile object.
 13. The information processing apparatus according to claim 12, wherein the autonomous mobile object includes a structure corresponding to a mouth, and the behavior planner is configured to, when the control mode of the autonomous mobile object is a silent mode, plan the behavior such that the structure corresponding to the mouth does not open.
 14. The information processing apparatus according to claim 1, wherein the behavior planner is configured to, based on changes in illuminance, plan the behavior satisfying at least one of self-preservation needs and self-esteem needs.
 15. The information processing apparatus according to claim 14, wherein the behavior planner is configured to, based on a surrounding environment or a condition of the user, determine a priority relating to the self-preservation needs or the self-esteem needs.
 16. The information processing apparatus according to claim 1, wherein the behavior planner is configured to, when it is detected that the user houses the autonomous mobile object, plan the behavior satisfying both self-preservation needs and self-esteem needs.
 17. The information processing apparatus according to claim 1, further comprising an operation controller configured to control operations of the autonomous mobile object based on the behavior planned by the behavior planner.
 18. The information processing apparatus according to claim 1, further comprising independent two display devices corresponding to eyeballs.
 19. An information processing method comprising: by a processor, planning a behavior of an autonomous mobile object based on estimation of circumstances, wherein the planning includes, based on the circumstances that are estimated and multiple sets of needs that are opposed to each other, determining the behavior to be executed by the autonomous mobile object.
 20. A program for causing a computer to function as an information processing apparatus comprising a behavior planner configured to plan a behavior of an autonomous mobile object based on estimation of circumstances, wherein the behavior planner is configured to, based on the circumstances that are estimated and multiple sets of needs that are opposed to each other, determine the behavior to be executed by the autonomous mobile object. 